[Security Solution] Bulk add alerts to chat#270286
Conversation
022553c to
942afc3
Compare
|
Pinging @elastic/security-threat-hunting (Team:Threat Hunting) |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#12417[✅] x-pack/solutions/security/plugins/security_solution/test/scout/ui/parallel.playwright.config.ts (--arch stateful --domain classic): 25/25 tests passed. |
|
test plan available here |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#12420[✅] x-pack/solutions/security/plugins/security_solution/test/scout/ui/parallel.playwright.config.ts (--arch stateful --domain classic): 50/50 tests passed. |
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
a04d943 to
eedc128
Compare
… resolveMaxContentLength Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… tool schema
The conversation attachment XML used id="..." as the attribute name, but
the attachment_read tool (registered as attachments_read) requires
attachment_id as its parameter. The instruction text also referenced
attachment_read(id) rather than attachment_read(attachment_id).
Together these caused the LLM to call attachments_read({ id: "..." }),
which Zod silently strips (strip mode), leaving attachment_id as
Required — resulting in a 500 on every attachment_read attempt.
Fix:
- Rename the XML attribute from id to attachment_id in both inline and
summary mode so the attribute name directly matches the tool parameter
- Update all instruction text references from attachment_read(id) to
attachment_read(attachment_id)
- Update the security alerts agent description to reflect the renamed
attribute and explicitly name the tool (attachments_read)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ge with bulk alerts attachment_read spec Moves the bulk_alerts_attachment_read eval out of the platform agent-builder suite and into a dedicated security-owned suite. Restores trace-based evaluators (latency, token counts, tool calls) and registers the suite in evals.suites.json for Buildkite CI via the evals:security-alert-triage label. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…nthetic datasets Adds alert_triage_quality.spec.ts with two LLM criteria-judged scenarios: priority triage (1 critical buried among 99 lower-severity alerts) and entity correlation (20 alerts on web-server-01 among 80 on other hosts). Synthetic alerts are seeded into .alerts-security.alerts-default in beforeAll with refresh:'wait_for' and cleaned up in afterAll. Setup failures throw immediately rather than logging a warning. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tachment - Bound alertIds to [1,20] with Zod .min(1).max(20); empty batches now rejected at validation rather than producing a zero-alert attachment - Add server-side guard in getAlertsById throwing when ids.length > 20, preventing unbounded ES terms queries regardless of call site - Remove try/catch around resolveMaxContentLength so resolution errors propagate visibly instead of silently truncating to the 10k default - Fix agent description tool name: attachments_read → attachment_read, consistent with the framework system prompt - Fix misleading JSDoc on maxContentLength: remove "highest value wins" claim; limit is applied per-attachment, not across types Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ip is removed Adds optional `groupId` to `AttachmentInput` and `Attachment`. All batches produced by `alertsToAttachmentInputs` share a single UUID so that removing the visible chip also removes its hidden siblings. Falls back to index-based removal for attachments without a groupId, preserving existing behaviour. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces AttachmentGroup as a first-class client-side entity
({ type: 'group', id, label, items: AttachmentInput[] }) replacing the
groupId + hidden flag shim. ConversationAttachment is a discriminated
union of AttachmentInput | AttachmentGroup; flattenAttachments converts
it to AttachmentInput[] at the single serialization boundary in
use_send_message_mutation.ts. Removes groupId from the server schema.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…tachments Renders AttachmentGroup entities as a single labelled chip in the conversation input. attachment_pills_row.tsx discriminates on isAttachmentGroup and renders AttachmentGroupPill for groups and the existing AttachmentPill for individual attachments. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…s to AttachmentGroup alertsToAttachmentGroup now returns a single AttachmentGroup with N batches in items (no groupId, no hidden, no attachmentLabel). A new useBulkAddToChatConfig hook centralises convertAlertToAttachment for all four alert table surfaces (alerts page, rule details, alert summary, attack discovery, cases). BulkAlertPathway union type is exported for telemetry. get_alerts_by_id refactored to return Record<id, source>. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add bounded module-level cache (500 entries, LRU eviction) so getRepresentation() re-fetches from ES at most once per attachment ID - Use getAlertsIndex(spaceId) instead of manual index string construction - Use undefined sentinel for alertsById instead of fetchFailed flag to prevent duplicate warn logging on error - Use explicit undefined checks instead of non-null assertions - Add unique attachment IDs in tests to avoid cross-test cache hits; add cache-hit test verifying getStartServices is not called on repeat reads Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…upPill Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ze limit introduced for bulk alert attachments
…hmentReferences When bulk-adding alerts the chat route chunked them into multiple AttachmentInput batches, causing one "Attachment added:" line per batch instead of one per logical group. - Add group_id and description to Attachment/VersionedAttachment/AttachmentInput types and propagate through server validation, state manager, and chat route - Stamp group_id and description on group items at the flattenAttachments boundary (the sole serialisation point), not at call sites - Deduplicate by group_id in RoundAttachmentReferences; actor-filter runs before the seenGroupIds slot is consumed so a non-matching actor ref cannot silently block a matching one from rendering - Fix description falsy-check in AttachmentStateManager (use !== undefined) - Add tests for RoundAttachmentReferences dedup logic (incl. actor-filter regression), flattenAttachments group stamping, and buildOptimisticAttachments group_id/description passthrough Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
126c654 to
0602f08
Compare
|
Thanks for the feedback @ryankeairns! I've used the existing styles for attachments and I think it is probably out of scope to change the general attachment styling in this PR. Here are some other examples of how attachments are styled today: |
seanrathier
left a comment
There was a problem hiding this comment.
Approving @elastic/contextual-security-apps ownership not to block
Nothing for us to review
…stem (#270903) # Summary This PR is extracted from [#270286](#270286) as the first of two stacked PRs. This first PR contains the needed changes in Agent Builder to support adding several attachments as a single batch to the chat. This followup [PR](#270904) adds the possibility bulk add Alerts to the chat. ## What This PR introduces `AttachmentGroup` as a client-side grouping primitive and `group_id` as a first-class optional field on `Attachment`, `VersionedAttachment`, and `AttachmentInput`. The group concept is dissolved at the `flattenAttachments` serialization boundary — individual items are stamped with `group_id` — and the full pipeline (server routes, state manager, agent execution) threads the field through to persistence and presentation. ## Why Enables consumers (e.g. the Security Solution bulk-alerts feature) to attach a batch of items as a single logical group, have them deduplicated to one chip in chat history, and removed atomically. The [follow-up PR](#270904) shows the first concrete use: bulk-adding alerts to chat. Without this platform layer, each consumer would have to roll its own grouping and dedup logic. Per-attachment `maxContentLength` is introduced at the same time because bulk alert documents (full ES source) can be large. A single global truncation limit would either under-truncate small attachments or over-truncate large ones. Delegating the limit to the attachment type registration lets each type declare its own budget independently. Closes: elastic/security-team#17544 Epic: elastic/security-team#17311 ## Changes **Types and contracts** - `AttachmentGroup` type + `isAttachmentGroup` predicate added to `versioned_attachment.ts` - `group_id` and `description` optional fields added to `Attachment`, `AttachmentInput`, and `VersionedAttachment` - `plugin_contract.ts` exports updated to include `AttachmentGroup` - `maxContentLength?: number` added to `AttachmentTypeDefinition` — per-type inline truncation limit (characters), defaults to 10 000 if absent. **Client-side logic** - `flatten_attachments.ts` — new canonical serialization boundary; stamps `group_id` and `description` on group items - `remove_attachment_from_list.ts` — handles group-id-based bulk removal (removes all items sharing a `group_id`) - `upsert_attachments_into_list.ts` — updated to handle `AttachmentGroup` alongside `AttachmentInput` - `build_optimistic_attachments.ts` — preserves `group_id` and `description` on fallback attachments **UI** - `attachment_group_pill.tsx` — new chip component for rendering an `AttachmentGroup` in the input row (same visual style as `AttachmentPill`) - `attachment_pills_row.tsx` / `conversation_input.tsx` — wired to render `AttachmentGroupPill` for groups - `round_attachment_references.tsx` — deduplicates attachment refs by `group_id` (actor-filter check now correctly runs before the `seenGroupIds` slot is consumed) **Server pipeline** - `chat.ts` — accepts and forwards `group_id` / `description` from request body - `validate_attachment.ts` — passes `group_id` and `description` through to state manager - `attachment_state_manager.ts` — persists `group_id` to `VersionedAttachment` - `prepare_conversation.ts` / `attachment_presentation.ts` — threads `group_id` and per-attachment `maxContentLength` through to agent execution - `attachment_presentation.ts` — replaces the flat `maxContentLength` config option with `resolveMaxContentLength?: (attachment) => number | undefined`, a per-attachment resolver. `prepare_conversation.ts` wires this to `attachmentsService.getTypeDefinition(type)?.maxContentLength` so any attachment type can declare its own inline truncation limit via its type registration. - XML attribute `id` renamed to `attachment_id` in inline and summary attachment nodes, and in the LLM instructions, to remove ambiguity. **Tests** - New: `flatten_attachments.test.ts`, `remove_attachment_from_list.test.ts`, `versioned_attachment.test.ts`, `round_attachment_references.test.tsx` - Updated: `upsert_attachments_into_list.test.ts`, `build_optimistic_attachments.test.ts`, `attachment_presentation.test.ts` ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks - **Data loss / corruption:** Low. `group_id` and `description` are optional fields appended to `VersionedAttachment`. Existing saved conversations have neither field; all read paths guard on `!== undefined`, so no migration is needed and existing data is unaffected. - **Regressions:** Low. `AttachmentGroup` has no producers until the follow-up security PR lands. All existing `AttachmentInput` paths are unchanged. The `flattenAttachments` function is additive — it handles both plain `AttachmentInput` and `AttachmentGroup` and existing call sites have been updated. - **Performance:** Negligible. The only new overhead is a `Set`-based group-id dedup during `RoundAttachmentReferences` rendering. - **Hard-to-test:** The actor-filter / `seenGroupIds` ordering edge case (filter must run before consuming the Set slot) is subtle but is now covered by a dedicated test. ### Release note > No user-facing changes. **Suggested label:** `release_note:skip` --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…stem (elastic#270903) # Summary This PR is extracted from [elastic#270286](elastic#270286) as the first of two stacked PRs. This first PR contains the needed changes in Agent Builder to support adding several attachments as a single batch to the chat. This followup [PR](elastic#270904) adds the possibility bulk add Alerts to the chat. ## What This PR introduces `AttachmentGroup` as a client-side grouping primitive and `group_id` as a first-class optional field on `Attachment`, `VersionedAttachment`, and `AttachmentInput`. The group concept is dissolved at the `flattenAttachments` serialization boundary — individual items are stamped with `group_id` — and the full pipeline (server routes, state manager, agent execution) threads the field through to persistence and presentation. ## Why Enables consumers (e.g. the Security Solution bulk-alerts feature) to attach a batch of items as a single logical group, have them deduplicated to one chip in chat history, and removed atomically. The [follow-up PR](elastic#270904) shows the first concrete use: bulk-adding alerts to chat. Without this platform layer, each consumer would have to roll its own grouping and dedup logic. Per-attachment `maxContentLength` is introduced at the same time because bulk alert documents (full ES source) can be large. A single global truncation limit would either under-truncate small attachments or over-truncate large ones. Delegating the limit to the attachment type registration lets each type declare its own budget independently. Closes: elastic/security-team#17544 Epic: elastic/security-team#17311 ## Changes **Types and contracts** - `AttachmentGroup` type + `isAttachmentGroup` predicate added to `versioned_attachment.ts` - `group_id` and `description` optional fields added to `Attachment`, `AttachmentInput`, and `VersionedAttachment` - `plugin_contract.ts` exports updated to include `AttachmentGroup` - `maxContentLength?: number` added to `AttachmentTypeDefinition` — per-type inline truncation limit (characters), defaults to 10 000 if absent. **Client-side logic** - `flatten_attachments.ts` — new canonical serialization boundary; stamps `group_id` and `description` on group items - `remove_attachment_from_list.ts` — handles group-id-based bulk removal (removes all items sharing a `group_id`) - `upsert_attachments_into_list.ts` — updated to handle `AttachmentGroup` alongside `AttachmentInput` - `build_optimistic_attachments.ts` — preserves `group_id` and `description` on fallback attachments **UI** - `attachment_group_pill.tsx` — new chip component for rendering an `AttachmentGroup` in the input row (same visual style as `AttachmentPill`) - `attachment_pills_row.tsx` / `conversation_input.tsx` — wired to render `AttachmentGroupPill` for groups - `round_attachment_references.tsx` — deduplicates attachment refs by `group_id` (actor-filter check now correctly runs before the `seenGroupIds` slot is consumed) **Server pipeline** - `chat.ts` — accepts and forwards `group_id` / `description` from request body - `validate_attachment.ts` — passes `group_id` and `description` through to state manager - `attachment_state_manager.ts` — persists `group_id` to `VersionedAttachment` - `prepare_conversation.ts` / `attachment_presentation.ts` — threads `group_id` and per-attachment `maxContentLength` through to agent execution - `attachment_presentation.ts` — replaces the flat `maxContentLength` config option with `resolveMaxContentLength?: (attachment) => number | undefined`, a per-attachment resolver. `prepare_conversation.ts` wires this to `attachmentsService.getTypeDefinition(type)?.maxContentLength` so any attachment type can declare its own inline truncation limit via its type registration. - XML attribute `id` renamed to `attachment_id` in inline and summary attachment nodes, and in the LLM instructions, to remove ambiguity. **Tests** - New: `flatten_attachments.test.ts`, `remove_attachment_from_list.test.ts`, `versioned_attachment.test.ts`, `round_attachment_references.test.tsx` - Updated: `upsert_attachments_into_list.test.ts`, `build_optimistic_attachments.test.ts`, `attachment_presentation.test.ts` ### Checklist - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [x] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels. ### Identify risks - **Data loss / corruption:** Low. `group_id` and `description` are optional fields appended to `VersionedAttachment`. Existing saved conversations have neither field; all read paths guard on `!== undefined`, so no migration is needed and existing data is unaffected. - **Regressions:** Low. `AttachmentGroup` has no producers until the follow-up security PR lands. All existing `AttachmentInput` paths are unchanged. The `flattenAttachments` function is additive — it handles both plain `AttachmentInput` and `AttachmentGroup` and existing call sites have been updated. - **Performance:** Negligible. The only new overhead is a `Set`-based group-id dedup during `RoundAttachmentReferences` rendering. - **Hard-to-test:** The actor-filter / `seenGroupIds` ordering edge case (filter must run before consuming the Set slot) is subtle but is now covered by a dedicated test. ### Release note > No user-facing changes. **Suggested label:** `release_note:skip` --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Summary
What
Adds a bulk action to the security alerts table that lets analysts select multiple alerts and send them to the AI Assistant chat in a single click. Alert data is fetched server-side at render time and delivered inline to the LLM — no extra round-trips for small-to-medium selections. Large selections (>100 alerts, i.e. >5 batches) fall back to summary mode where the LLM reads batches on demand via
attachment_read.Why
Analysts need to triage, prioritise, and find patterns across multiple alerts together in the AI Assistant. Previously only single-alert add-to-chat was supported.
Closes elastic/security-team#17496
Epic: elastic/security-team#17311
bulk-add-alerts2.mov
Changes
security.alertsattachment type (server)attachment_readcalls during a conversationAttachmentGroupandAttachmentGroupPill(agent-builder platform plugin)AttachmentGroupis a named collection of relatedAttachmentInputitems that share a stablegroup_id;AttachmentGroupPillrenders the group as one labelled pill (e.g. "20 Alerts") in the conversation input — removing the pill atomically removes all grouped itemsflattenAttachmentsis the sole serialisation boundary where groups dissolve intoAttachmentInput[]; it stampsgroup_idanddescriptiononto each item so group membership survives the server round-tripgroup_idis a first-class field onAttachment,VersionedAttachment, andAttachmentInput, persisted server-side and returned in conversation responsesRoundAttachmentReferencesdeduplicates bygroup_idso all batches from a single bulk-add action appear as one "Attachment added: N Alerts" line in chat historyupsert_attachments_into_listandremove_attachment_from_listhandle groups atomicallyBulk action wired into all four alert table surfaces (security_solution public)
alertsToAttachmentGroupconverts selected alert items into a singleAttachmentGrouplabelled "N Alerts", chunked into batches of ≤20use_bulk_add_to_chat_confighook centralises config and telemetry per surfaceShared alerts-table package (
response-ops/alerts-table)BulkAddToChatConfigandOpenChatServiceinterfaces intypes.tswith no compile-time dependency on agent-builder packagesuseBulkAddToChatActionshook wired intoAlertsDataGridAgent Builder server/common
maxContentLengthonAttachmentTypeDefinition— per-attachment content cap replacing the previous global ceilinggetAlertsByIdutility for the server-side ES fetchprepareAttachmentPresentationthreads the per-attachmentresolveMaxContentLengthresolver throughalert_countfield in theAddToChatEBT telemetry eventTests
RoundAttachmentReferences,flattenAttachments, andbuildOptimisticAttachmentseach have tests covering group dedup, actor-filter edge cases, andgroup_id/descriptionpassthroughbulk_add_alerts_to_chat.spec.ts) covering the end-to-end flow on the alerts pagekbn-evals-suite-security-alert-triageeval package registered in the evals pipeline:bulk_alerts_attachment_read.spec.ts: deterministic compliance eval verifying the LLM callsattachment_readfor every batch in summary modealert_triage_quality.spec.ts: LLM-as-judge quality eval with two 100-alert synthetic scenarios — priority triage with a buried critical alert, and entity correlation identifying a repeatedly targeted hostChecklist
release_note:breakinglabel should be applied in these situations.release_note:*label is applied per the guidelinesbackport:*labels.Identify risks
agentBuilderServiceandbulkAddToChatConfigare optional props — if absent the bulk action simply does not appear; no existing actions are modified.termsfilter on_id. No full scans; scoped to the user's space index. Representation cache avoids redundant fetches within a conversation.maxContentLength: 50_000per batch caps inline content. The framework automatically switches to metadata-only summary mode for >5 attachments (>100 alerts), with the LLM reading batches on demand viaattachment_read.format()is unit-tested with a mock client. Consistent with how other security attachment types are tested.Release note
Suggested label:
release_note:feature